Skip to content

Adds 'redis' WORKER_TYPE#7296

Merged
dkliban merged 1 commit intopulp:mainfrom
dkliban:7210
Apr 7, 2026
Merged

Adds 'redis' WORKER_TYPE#7296
dkliban merged 1 commit intopulp:mainfrom
dkliban:7210

Conversation

@dkliban
Copy link
Copy Markdown
Member

@dkliban dkliban commented Feb 8, 2026

This adds WORKER_TYPE setting. The default value is 'pulpcore'. When 'redis' is selected, the tasking system uses Redis to lock resources. Redis workers produce less load on the PostgreSQL database.

closes: #7210

Generated By: Claude Code.

📜 Checklist

  • Commits are cleanly separated with meaningful messages (simple features and bug fixes should be squashed to one commit)
  • A changelog entry or entries has been added for any significant changes
  • Follows the Pulp policy on AI Usage
  • (For new features) - User documentation and test coverage has been added

See: Pull Request Walkthrough

Copy link
Copy Markdown
Contributor

@gerrod3 gerrod3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's still a lot I haven't deeply reviewed yet, but this was getting long and I had a big idea around dispatch that I want to discuss

Copy link
Copy Markdown
Contributor

@gerrod3 gerrod3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lots of comment/logging statements to remove. Need more specificity on the try/except blocks. And finally there are gaps in the task logic that need to be addressed.

@dkliban dkliban force-pushed the 7210 branch 3 times, most recently from 2ac9198 to 8d4de21 Compare March 25, 2026 10:21
Copy link
Copy Markdown
Contributor

@gerrod3 gerrod3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nits, but everything else looks good. Think these should be the last changes needed.

Comment on lines +544 to +545
@pytest.mark.parallel
def test_immediate_task_execution_in_worker(dispatch_task, monitor_task):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These following three tests seem pointless, let's remove them.

Comment on lines +667 to +668
@pytest.mark.parallel
def test_immediate_task_with_available_resources(dispatch_task, pulpcore_bindings):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same with this test, it is already tested below, let's remove.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test specifically exercises the are_resources_available() / acquire_locks() code path that the no-resource test skips entirely.

Copy link
Copy Markdown
Contributor

@gerrod3 gerrod3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is ready. Thanks for all the work!

This adds WORKER_TYPE setting. The default value is 'pulpcore'. When 'redis' is selected,
the tasking system uses Redis to lock resources. Redis workers produce less load on the
PostgreSQL database.

closes: pulp#7210

Generated By: Claude Code.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@dkliban dkliban merged commit eb12053 into pulp:main Apr 7, 2026
13 of 14 checks passed
@dkliban dkliban deleted the 7210 branch April 7, 2026 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a Redis based worker to the tasking system

2 participants